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fSJ , Abstract 



Cyber-secure networked control is modeled, analyzed, and experimentally illustrated in this paper. An attack space defined 

■ by the adversary's system knowledge, disclosure, and disruption resources is introduced. Adversaries constrained by these 
resources are modeled for a networked control system architecture. It is shown that attack scenarios corresponding to denial- 
of-service, replay, zero-dynamics, and bias injection attacks can be analyzed using this framework. Furthermore, the attack 

■ policy for each scenario is described and the attack's impact is characterized using the concept of safe sets. An experimental 
setup based on a quadruple-tank process controlled over a wireless network is used to illustrate the attack scenarios, their 
consequences, and potential counter-measures. 
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■ 1 Introduction 

y—{ ' Safe and reliable operation of infrastructures is of major societal importance. These systems need to be engineered 
fSl ■ in such a way so that they can be continuously monitored, coordinated, and controlled despite a variety of poten- 
tial system disturbances. Given the strict operating requirements and system complexity, such systems are operated 
through IT infrastructures enabling the timely data flow between digital controllers, sensors, and actuators. However, 
. , , . the use of communication networks and heterogeneous IT components has made these cyber-physical systems vul- 

■ nerable to cyber threats. One such example are the industrial systems and critical infrastructures operated through 
5-H ] Supervisory Control and Data Acquisition (SCADA) systems. The measurement and control data in these systems 
5^ . are commonly transmitted through unprotected communication channels, leaving the system vulnerable to several 

threats [10]. As illustrative examples, we mention the cyber attacks on power transmission networks operated by 
SCADA systems reported in the public media [11], and the Stuxnet malware that supposedly infected an industrial 
control system and disrupted its operation [9,21]. 



There exists a vast literature on computer security focusing on three main properties of data and IT services, namely 
confidentiality, integrity, and availability [3]. Confidentiality relates to the non-disclosure of data by unauthorized 
parties. Integrity on the other hand concerns the trustworthiness of data, meaning there is no unauthorized change 
of the data contents or properties, while availability means that timely access to the data or system functionalities 
is ensured. Unlike other IT systems where cyber-security mainly involves the protection of data, cyber attacks on 
networked control systems may influence physical processes through feedback actuation. Therefore networked control 
system security needs to consider threats at both the cyber and physical layers. Furthermore, it is of the utmost 
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importance in the study of cyber attacks on control systems to capture the adversary's resources and knowledge. 
Cyber threats can be captured in the attack space illustrated in Figure 1, which depicts several attack scenarios as 
points. For instance, the eavesdropping attack and the denial-of-service (DoS) attack are indicated in the figure. 



System knowledge 




Covert attack 



Disclosure resources 



Disruption resources 



Figure 1. The cyber-physical attack space. 

We propose three dimensions for the attack space; the adversary's a priori system model knowledge and his disclosure 
and disruption resources. The a priori system knowledge can be used by the adversary to construct more complex 
attacks, possibly harder to detect and with more severe consequences. Similarly, the disclosure resources enable the 
adversary to obtain sensitive information about the system during the attack by violating data confidentiality. Note 
that disclosure resources alone cannot disrupt the system operation. An example of an attack using only disclosure 
resources is the eavesdropping attack illustrated in Figure 1. On the other hand, disruption resources can be used to 
affect the system operation, which happens for instance when data integrity or availability properties are violated. 
One such example is the DoS attack, where the data required for correctly operating the system are made unavailable. 
In particular this characterization fits the Stuxnet malware, which had resources to record and manipulate data in 
the SC'ADA network [9]. Moreover, the complexity and operation of Stuxnet also indicate that its developers had 
access to a reasonable amount of knowledge of both physical and cyber components of the target control system. 

1.1 Related Work 

Control theory has contributed with frameworks to handle model uncertainties and disturbances as well as fault 

diagnosis and mitigation, see, for example, [32] and [7, 14], respectively. These tools can be used to detect and 
attenuate the consequences of cyber attacks on networked control systems, as has recently been done in the literature. 

Cyber attacks on control systems compromising measurement and actuator data integrity and availability have been 
considered in [6], where the authors modeled the attack effects on the physical dynamics. Several attack scenarios 

have been simulated and evaluated on the Tennessee-Eastman process control system [5] to study the attack impact 
and detectability. The attack scenarios in [5] are related to the ones considered in this paper, but we quantify the 
attack resources and policies in a systematic way. 

Availability attacks have been analyzed in [1, 12] for resource constrained adversaries with full-state information. 
Particularly, the authors considered DoS attacks in which the adversary could tamper with the communication 
channels and prevent measurement and actuator data from reaching their destination, rendering the data unavailable. 
A particular instance of the DoS attack in which the adversary does not have any a priori system knowledge, as the 
attack in [1], is represented in the attack space in Figure 1. 

Deception attacks compromising integrity have recently received attention. Replay attacks on the sensor measure- 
ments, which is a particular kind of deception attack, have been analyzed in [19]. The authors considered the case 
where all the existing sensors were attacked and suitable counter-measures to detect the attack were proposed. In 
this attack scenario the adversary docs not have any system knowledge but is able to access and corrupt the sensor 
data, thus having disclosure and disruptive resources, as depicted in Figure 1. 

Another class of deception attacks, false-data injection attacks, has been studied in recent work. For instance, in the 
case of power networks, an adversary with perfect model knowledge has been considered in [18]. The work in [17] 
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considered stealthy attacks with Umitcd resources and proposed improved detection methods, while [22] analyzed 
the minimum number of sensors required for stealthy attacks. A corresponding measurement security metric for 
studying sets of vulnerable sensors was proposed in [22] . The consequences of these attacks have also been analyzed 
in [26,28,30]. In particular, in [26] the authors analyzed attack policies with limited model knowledge and performed 
experiments on a power system control software, showing that such attacks are stealthy and can induce the erroneous 
belief that the system is at an unsafe state. The models used in the previous work are static, hence these attack 
scenarios are closest to the bias injection attack shown in Figure 1. 

Data injection attacks on dynamic control systems were also considered. In [25] the author characterizes the set of 
attack policies for covert (undetectable) false-data injection attacks with detailed model knowledge and full access 
to all sensor and actuator channels, while [20] described the set of undetectable false-data injection attacks for 
omniscient adversaries with full-state information, but possibly compromising only a subset of the existing sensors 
and actuators. In the context of multi-agent systems, optimal adversary policies for data injection using full model 
knowledge and state information were derived in [16]. In these attack scenarios confidentiality was violated, as the 
adversary had access to either measurement and actuator data or full-state information. These attacks are therefore 
placed close to the covert attack in Figure 1. 

Most of the recent work on cyber-security of control systems has considered scenarios where the adversary has access 
to a large set of resoiu-ces and knowledge, thus being placed far from the origin of the attack space in Figure 1. 
A large part of the attack space has not been addressed. In particular, the class of detectable attacks that do not 
trigger conventional alarms has yet to be covered in depth. 

1.2 Contributions and Outline 

In this paper we consider a typical networked control architecture under both cyber and physical attacks. A generic 
adversary model applicable to several attack scenarios is discussed and the attack resources are mapped to the 
corresponding dimensions of the attack space. To illustrate the proposed framework, we consider several attack 
scenarios where the adversary's goal is to drive the system to an unsafe state while remaining stealthy. For each 
scenario we formulate the corresponding stealthy attack policy, comment on the attack's performance, and describe 
the adversary's capabilities along each dimension of the attack space in Figure 1, namely the disclosure resources, 
disruption resources, and system knowledge. Some of the attack scenarios analyzed in the paper have been staged 
on a wireless quadruple tank testbed for security of control systems. The testbed architecture and results from the 
staged attacks are presented and discussed. 

One of the attack scenarios analyzed corresponds to a novel type of detectable attack, the bias injection attack. 
Although this attack may be detected, it can drive the system to an unsafe region and it only requires limited model 
knowledge and no information about the system state. Stealthiness conditions for this attack are provided, as well 
as a methodology to assess the attack impact on the physical state of the system. 

The material in this paper is an extension of the authors' preliminary work, see [27] . Particularly, in the current work 
the attack goals are formalized using the notion of safe regions of the state space and two additional attack scenarios 
are described and analyzed. Furthermore, the attack performance of each scenario is analyzed in more detail and 
additional results for the zero-dynamics and bias injection attacks are presented. 

The outline of the paper is as follows. The system architecture and model are described in Section 2, while Section 3 
contains the adversary model and a detailed description of the attack resources on each dimension of the attack 
space. The framework introduced in the previous sections is then illustrated for five particular attack scenarios in 
Section 4, supposing that the adversary aims at driving the system to an unsafe state while remaining stealthy. The 
attack policy, attack performance, and required system knowledge, disclosure, and disruption resources are described 
in detail for each attack scenario. The results of the experiments for four of the attack scenarios in a secure control 
systems testbed are presented and discussed in Section 5, followed by conclusions in Section 6. 

2 Networked Control System 

In this section we describe the networked control system structure, where we consider three main components: the 
physical plant and communication network, the feedback controller, and the anomaly detector. 
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2.1 Physical Plant and Communication Network 



The physical plant is modeled in a discrete-time state-space form 

^ _ f Xk+i = Axk + Buk + Gwk + Ffk 
' \ yk = Cxk +Vk ' 

where Xk G K" is the state variable, Uk G I^' the control actions applied to the process, yk G the measurements 
from the sensors at the sampling instant A; e Z, and fk S R** is the unknown signal representing the effects of 
anomalies, usually denoted as fault signal in the fault diagnosis literature [8]. The process and measurement noise, 
Wk G R" and Vk € K^, represent the discrepancies between the model and the real process, due to unmodeled 
dynamics or disturbances, for instance, and we assume their means are respectively bounded by 5^ and 5^, i.e. 
w = \\E{wk}\\ < and V = \\M{vk}\\ < Sy. 



The physical plant operation is supported by a communication network through which the sensor measurements and 
actuator data arc transmitted, which at the plant side correspond to yk and Uk, respectively. At the controller side 
we denote the sensor and actuator data by yk € MP and Uk € M', respectively. Since the communication network may- 
be unreliable, the data exchanged between the plant and the controller may be altered, resulting in discrepancies in 
the data at the plant and controller ends. In this paper we do not consider the usual communication network effects 
such as packet losses and delays. Instead wc focus on data corruption due to malicious cyber attacks, as described 
in Section 3. Therefore the communication network per se is supposed to be reliable, not affecting the data flowing 
through it. 

Given the physical plant model (1) and assuming an ideal communication network, the networked control system is 
said to have a nominal behavior if fk =Q,Uk = Uk, and yk = yk- The absence of either one of these condition results 
in an abnormal behavior of the system. 



2.2 Feedback Controller 



In order to comply with performance requirements in the presence of the unknown process and measurement noises, 
we consider that the physical plant is controlled by an appropriate linear time-invariant feedback controller [32] . The 
output feedback controller can be written in a state-space form as 



J, ^ ( Zk+i = AcZk + BcVk 
■ \ Uk = CcZk + Dc/yk 



(2) 



where the states of the controller, Zk G M™, may include the process state and tracking error estimates. Given the 
plant and communication network models, the controller is supposed to be designed so that acceptable performance 
is achieved under nominal behavior. 



2.3 Anomaly Detector 



In this section we consider the anomaly detector that monitors the system to detect possible anomalies, i.e. deviations 
from the nominal behavior. The anomaly detector is supposed to be collocated with the controller, therefore it only 
has access to yk and Uk to evaluate the behavior of the plant. 

Several approaches to detecting malfunctions in control systems are available in the fault diagnosis literature [8,14]. 
Here we consider the following observer-based Fault Detection Filter 

^ . f Xk\k = Axk-i\k-i + Buk-i + K{yk - yk\k-i) . . 

'I r-fe = V{yk - yk\k) 

where Xk\k € M" and yk\k = Cxk\k G are the state and output estimates given measurements up until time fc, 
respectively, and e V'^ the residue evaluated to detect and locate existing anomalies. 



The anomaly detector is designed by choosing K and V such that 
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Figure 2. Adversary model for a point in tlie attack space in Figure 1. 

(1) under nominal behavior of the system (i.e., fk = 0, Uk = Uk, Uk = Vk), the expected value of the residue 
converges asymptotically to a neighborhood of zero, i.e., limfe^oo ||]E{rjt;}|| < Sr, with Sr & K"'"; 

(2) the residue is sensitive to the anomalies {fk ^ 0). 



An alarm is triggered if the residue meets 

Ikfell > Sr + Sa, 

where 6a € M+ is chosen so that the false alarm rate does not exceed a given threshold a G [0, 1] . 



(4) 



3 Adversary Models 

The adversary model considered in this paper is illustrated in Figure 2 and is composed of an attack policy and 
the adversary resources i.e., the system model knowledge, the disclosure resources, and the disruption resources. 
Each of the adversary resources can be mapped to a specific axis of the attack space in Figure 1: K, = {V , T , V} 
is the a priori system knowledge possessed by the adversary; Xk corresponds to the set of sensor and actuator data 
available to the adversary at time k as illustrated in (8), thus being mapped to the disclosure resources; ak is the 
attack vector at time k that may affect the system behavior using the disruption resources captured by B, as defined 
in the current section. The attack policy mapping /C and Ik to Uk at time k is denoted as 



dk = 

Examples of attacks policies for different attack scenarios are given in Section 4. 



(5) 



In this section we describe the networked control system under attack with respect to the attack vector ak- Then 
we detail the adversary's system knowledge, the disclosure resources, and the disruption resources. Models of the 
attack vector ak for particular disruption resources are also given. 

3. 1 Networked Control System under Attack 

The system components under attack are now characterized for the attack vector ak, which also includes the fault 
signal fk- Considering the plant and controller states to be augmented as r]k = [xj. ^ the dynamics of the 

closed-loop system composed by V and T under the effect of ak can be written as 



where the system matrices are 



r}k+i = Arik + Bofe + G 
Vk = CT]k + Dafe + H 



A = 



A + BDcC BCc 
B,C A, 



, G = 



Wk 
Vk_ 

Wk 
Vk 



G BDc 
Be 



(6) 



C 



H 



/ 
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and B and D capture the way in which the attack vector affects the plant and controller. These matrices are 
characterized for some attack scenarios in Section 3.4. 



Similarly, using V and T) as in (1) and (3), respectively, the anomaly detector error dynamics under attack are 
described by 



Cfe|fe = Ae^fe_i|fe-l + Bettfe-i + Ge 
Tk = Ce^fe-l|fe-l + Deflfe-l + He 

where £,k\k € I^" is the estimation error and 



Wk-l 
Vk 

Wk-l 
Vk 



(7) 



Ae = (7 - KC)A, Ge = 
Ce = VC{I - KC)A, He = 



(7 - KC)G -K 
VC{I - KC)G V{I - CK) 



The matrices Be and De are specific to the available disruptive resources and are characterized in Section 3.4. 

3.2 System Knowledge 

The amount of a priori knowledge regarding the control system is a core component of the adversary model, as 
it may be used, for instance, to render the attack undetectable. In general, we may consider that the adversary 
approximately knows the model of the plant [V) and the algorithms used in the feedback controller {T) and the 
anomaly detector (2?), thus denoting the adversary knowledge by /C = {T', J^, I?}. Figure 1 illustrates several types 
of attack scenarios with different amounts of required system knowledge. In particular, note that the replay attacks 
do not need any knowledge of the system components, thus having /C = 0, while the covert attack requires full 
knowledge about the system, hence K, = {V,F,V}. 

3.3 Disclosure Resources 



The disclosure resources enable the adversary to gather sequences of data from the calculated control actions Uk and 
the real measurements yk through disclosure attacks. Denote TV^ C {1. . . . , g} and TZ^ C {1, . . . ,p} as the disclosure 
resources, i.e. set of actuator and sensor channels that can be accessed during disclosure attacks, and let Xk be the 
control and measurement data sequence gathered by the adversary from time fco to k. The disclosure attacks can 
then be modeled as 



Ik ■= Ik-i U 







' 




Uk 


1 




_ 











(8) 



where T" e BI^"Ix9 and Jy e ffil^^lxp are the binary incidence matrices mapping the data channels to the corre- 
sponding data gathered by the adversary and Ikg = 0. 



As seen in the above description of disclosure attacks, the physical dynamics of the system are not affected by these 
type of attacks. Instead, these attacks gather intelligence that may enable more complex attacks, such as the replay 
attacks depicted in Figure 1. 



3.4 Disruption Resources 



As seen in the system dynamics under attack, (6) and (7), disruption resources are related to the attack vector Uk 
and may be used to affect the several components of the system. The way a particular attack disturbs the system 
operation depends not only on the respective resources, but also on the nature of the attack. For instance, a physical 
attack directly perturbs the system dynamics, whereas a cyber attack disturbs the system through the cyber-physical 
couplings. To better illustrate this discussion we now consider physical and data deception attacks. 



6 



3.4-1 Physical Resources 



Physical attacks may occur in control systems, often in conjunction with cyber attacks. For instance, in [2] water was 
pumped out of an irrigation system while the water level measurements were corrupted so that the attack remained 
stealthy. Since physical attacks are similar to the fault signals fk in (1), in the following sections we consider fk to 
be the physical attack modifying the plant dynamics as 

Xk+i = Axk + Buk + Gwk + Ffk 
Uk = Cxk- 



Considering ak = fk, the resulting system dynamics are described by (6) and (7) with 

, D = 0, Be = {I-KC)F, T>e = VC{I - KC)F. 



B = 



Note that the disruption resources in this attack are captured in the matrix F. 
3.4-2 Data Deception Resources 

The deception attacks modify the control actions Uk and sensor measurements pk from their calculated or real values 
to the corrupted signals Uk and ijk, respectively. Denoting TZJ C {1, . . . , g} and TVj C {1, . . . ,p} as the deception 
resources, i.e. set of actuator and sensor channels that can be affected, the deception attacks are modeled as 



(9) 



where the signals € RI'^"! and bl G RI'^^I represent the data corruption and T" G B«^l^"l and r« e fiP^I^?! 
(B := {0, 1}) are the binary incidence matrices mapping the data corruption to the respective data channels. The 
matrices F" and F^ indicate which data channels can be accessed by the adversary and are therefore directly related 
to the adversary resources in deception attacks. 



Defining ak = [6^ K+i ] ' system dynamics are given by (6) and (7) with 



B = 



BT'' BD^ry 
B^ry 



D 



ory 



B, 



(/ - KC)BT'' -KVy 



VC{I - KC)BT" V{I - CK)Ty 



Note that deception attacks do not possess any disclosure capabilities, as depicted in Figure 1 for examples of 
deception attacks such as the bias injection attack. 



4 Attack Scenarios 

In this section we discuss the general goal of an adversary and likely choices of the attack policy g{-, •). In particular, 

using the framework introduced in the previous sections, we consider several attack scenarios where the adversary's 
goal is to drive the system to an unsafe state while remaining stealthy. For each scenario we formulate the cor- 
responding stealthy attack policy, comment on the attack's performance, and describe the adversary's capabilities 
along each dimension of the attack space in Figure 1, namely the disclosure resources, disruption resources, and 
system knowledge. A set of these scenarios is illustrated by experiments on a process control testbed in Section 5. 

4.1 Attack Goals and Constraints 

In addition to the attack resources, the attack scenarios need to also include the intent of the adversary, namely the 
attack goals and constraints shaping the attack policy. The attack goals can be stated in terms of the attack impact 
on the system operation, while the constraints may be related to the attack detectability. 
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Several physical systems have tight operating constraints which if not satisfied might result in physical damage to 
the system. In this work we use the concept of safe regions to characterize the safety constraints. 



Definition 1 At a given time instant k, the system is said to be safe if Xk € Sx, where is a closed and compact 
set with non-empty interior. 

Assumption 2 The system is in a safe state at the beginning of the attack, i.e. Xk^ € S^. 

The physical impact of an attack can be evalnatc^d by assc^ssing whether or not the state of the system remained in 
the safe set diiring and after the attack. The attack is considered successful if the state is driven out of the safe set. 

Regarding the attack constraints, we consider that attacks are constrained to remain stealthy. Furthermore, we 
consider the disruptive attack component consists of only physical and data deception attacks, and thus we have 
the attack vector = [fj ^^k+i ^V^V^ ■ Given the anomaly detector described in Section 2 and denoting 

A^l = {ctfeo , • • • J (ikf } as the attack signal, the set of stealthy attacks are defined as follows. 

Definition 3 The attack signal A,^'^ is stealthy if \\rk\\ < Sr + Sa, Vfc > fco- 

Note that the above definition is dependent on the initial state of the system at ko, as well as the noise terms Wk 
and Vk- 

Since the closed-loop system (6) and the anomaly detector (7) under linear attack policies are linear systems, each 
of these systems can be separated into two components, the nominal component with a/j = Vfc and the following 

systems 

y^ = Cr7^ + Da, ^'^^ 

and 

^k\k = Ae^fe_i|fe_i + Bettfe-i ^^^^ 
rt = CeCfe_i|fc_i + Deflfe-l, 

with r,S = ^S\o = 0- 

Assuming the system is behaving nominally before the attack, using the triangle inequality and linearity of (7) we 
have ||r^|| < Sa => \ \rk\ \ < Sr + 5a, leading to the following definition: 

kf 

Definition 4 The attack signal ^j.^ is a— stealthy with respect toV ^/ ||rg|| < 5a, yk> ko. 

kf 

Albeit more conservative than Definition 3, this definition only depends on the attack signals Af.^ . Similarly, the 
impact of attacks on the closed-loop system can also be analyzed by looking at the linear system (10), as illustrated 
in Section 4.6 for the bias injection attack. 



4.2 Denial-of- Service Attack 



The DoS attacks prevent the actuator and sensor data from reaching their respective destinations and should therefore 
be modeled as the absence of data, for instance itfe = if all the actuator data is unavailable. However such a model 
would not fit the framework in (6) and (7) where ak is assumed to be a real valued vector. Hence we consider instead 
one of the typical mechanisms used by digital controllers to deal with the absence of data [23] , in which the absent 
data is replaced with the last received data, m^^^ and y-ry respectively. Denoting TV^ C {1, . . . ,q\ and TZ\ C {1, . . . ,p} 
as the set of actuator and sensor channels that can be made unavailable, we can model DoS attacks as deception 
attacks in (9) with 

hl:=-StT-^{uk-Ur^) 
bl :=-SyTy^{y,-y^J 

where S]i G bI^^I><I^^I and Si G bI^aIx|'^aI arc boolean diagonal matrices where the i—th diagonal entry indicates 

whether a DoS attack is performed ([<S'fe '']ii = 1) or not ([<S';^ '']ii = 0) on the corresponding channel. Therefore DoS 
attacks on the data are a type of disruptive attacks, as depicted in Figure 1. 



8 



Attack policy: The attack scenario analyzed in this paper considers a Bernoulli adversary [1] on the sensor channels 
following the random policy 

P([^fe]u = 1) = 0, vi = 1, . . . , \n\\, k<ko 

where p is the probability of blocking the data packet at any given time. 

Attack performance: Although the absence of data packets is not stealthy since it is trivially detectable, DoS 
attacks may be misdiagnosed as a poor network condition. As for the impact on the closed-loop system, the results 
available for Bernoulli packet losses readily apply to the current attack scenario [23,24,31]. In particular, we recall 
a result for the case where a hold scheme (12) is used in the absence of data. 

Proposition 5 (Theorem 8 in [31]) Assume the closed-loop system with no DoS attack is stable. Then the closed- 
loop system with Bernoulli DoS attacks is exponentially stable for p G [0, 1) if the open-loop system 



Vk+i 



A BCc 

Ar. 



Vk 



is marginally stable. 

Disclosure resources: Although the proposed model of DoS attacks in (12) contains the control and output 
signals, note that no disclosure resources are needed in the actual implementation of the attack. Thus we have 
W =W = 0. 

Disruption resources: The disruption capabilities correspond to the data channels that the adversary is able to 
make unavailable, TV^ and 7?.^. 

System knowledge: For the Bernoulli attack policy, no a priori knowledge of the system model is needed. 
4.3 Replay Attack 

In replay attacks the adversary first performs a disclosure attack from k = ko until kr, gathering sequences of data 
Tfe^, and then begins replaying the recorded data at time fc = fc^ + 1 until the end of the attack at fc = fc/ > A:^, as 
illustrated in Figure 3. In the scenario considered here the adversary is also able to perform a physical attack while 
replaying the recorded data, which covers the experiment on a water management SCADA system reported in [2] 
and one of Stuxnet's operation mode [9]. 
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(a) Phase I of the replay attack (13). (b) Phase II of the replay attack (14). 

Figure 3. Schematic of the replay attack. 



Attack policy: Similar to the work in [19], assuming TZ^'^ = TZ^j^ i.e., the adversary can corrupt the digital channels 
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from which the data sequences are gathered, the replay attack pohcy can be described as 



Phase I: 




(13) 



with ko < k < kr and I; 



ko 



and 



Phase II: < 



T"(Ufe-T - Uk) 
'^^{Vk+l-T - Uk+l) 
^'"{Vk-T - Vk) 
, I/s = Xfe_i, 



(14) 



where T = kr — I -\- k^ and kr + I < k < kf . An interesting instance of this attack scenario consists of applying a 
prc-dcfincd physical attack to the plant, while using replay attacks to render the attack stealthy. In this case the 
physical attack signal fk corresponds to an open- loop signal, fk = gf(k). 

Attack performance: The work in [19] provided conditions under which replay attacks with access to all measure- 
ment data channels are stealthy. However, these attacks are not guaranteed to be stealthy when only a subset of the 
data channels is attacked. In this case, the stealthiness constraint may require additional knowledge of the system 
model. For instance, the experiment presented in Section 5 requires knowledge of the physical system structure, 
so that fk only excites the attacked measurements. Hence fk can be seen as a zero-dynamics attack with respect 
to the healthy measurements, which is characterized in the section below. Since the impact of the replay attack is 
dependent only on fk, we refer the reader to Section 4.4 for a characterization of the replay attack's impact. 



Disclosure resources: The disclosm-e capabilities required to stage this attack correspond to the data channels 
that can be eavesdropped by the attacks, namely 7?." and W . 



Disruption resources: In this case the deception capabilities correspond to the data channels that the adversary 
can tamper with, TVj and TVj. In particular, for replay attacks the adversary can only tamper with the data channels 
from which data has been previously recorded, i.e. TV} C 7?." and 7?.j C W . 



Direct disruption of the physical system through the signal fk depends on having direct access to the physical system, 
modeled by the matrix F in (1). 



System knowledge: Note that no a priori knowledge /C on the system model is needed for the cyber component 
of the attack, namely the data disclosure and deception attack, as seen in the attack policy (13) and (14). As for 
the physical attack, fk, the required knowledge is scenario dependent. In the scenario considered in the experiments 
described in Section 5, this component was modeled as an open-loop signal, fk = 9f{k). 



4-4 Zero-Dynamics Attack 

Recalling that for linear attack policies the plant and the anomaly detector are linear systems, (10) and (11) 
respectively, Definition 4 states that this type of attacks are 0— stealthy if = 0, A; = ko,...,kf. The idea of 

0— stealthy attacks then consists of designing an attack policy and attack signal A'I^^ so that the residue rfc does not 

change due to the attack. 

A particular subset of 0— stealthy attacks are characterized in the following lemma: 

ke 

Lemma 6 The attack signal A^'^ is 0— stealthy with respect to any T> if = 0, Vfc > ko- 
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PROOF. Consider the attacked components of the controller and the anomaly detector in (10) and (11) with 



Xq = ^QtQ = 0. From the controller dynamics it directly follows that 



3| 

the input to the controller (y^) is zero. Since Xq 
are zero, we then conclude = 0, Vfc > fco- 



0, Vfc > fco results in 



0, VA; > ko, as 



and Vk — u% = 0, > ko, meaning that the detector's inputs 



Both the definition of 0— stealthy attacks and Lemma 6 indicate that these attacks are decoupled from the outputs of 
linear systems, rfe and yk respectively. Hence finding 0— stealthy attack signals relates to the output zeroing problem 
or zero-dynamics studied in the control theory literature [32] . Note that such an attack requires the perfect knowledge 

of the plant dynamics P and the attack signal is then based on the open-loop prediction of the output changes due 
to the attack. This is illustrated in Figure 4 where /C^ denote the zero-dynamics and there is no disclosure of sensor 
or actuator data. 



Uk 





V 
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Uk 
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V 



Vk 
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Figure 4. Schematic of the zero-dynamics attack. 

Attack policy: The attack policy then corresponds to the input sequence (a^) that makes the outputs of the 
process (y^) identically zero for all k and is illustrated in Figure 4. It can be shown [32] that the solution to this 
problem is given by the sequence 

ak = gy^, (15) 
parameterized by the input-zero direction g and the system zero u. 

For sake of simplicity we consider a particular instance of this attack, where only the actuator data is corrupted. In 
this case the zero attack policy corresponds to the transmission zero-dynamics of the plant. The plant dynamics due 
to an attack on the actuator data are described by 



Axl + Bak 



Cxi 



(16) 



with Uk = b^. Given the discrete-time system (16) with B having full column rank, the transmission zeros can be 
calculated as the values v G C that cause the matrix P{v) to lose rank, where 



vI-A-B 
C 



Those values are called minimum phase or non-minimum phase zeros depending on whether they are stable or 
unstable zeros, respectively. In discrete-time systems a zero is stable if [i^] < 1 and unstable otherwise. 

The input zero direction can be obtained by solving the following equation 



'vl - A -B 








"o" 


C 




_9 _ 








(17) 



where xq is the initial state of the system for which the input sequence (15) results in an identically zero output. 
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Lemma 7 Let xq be the initial state of the system, where xq satisfies (17). The state trajectories generated by the 
zero- dynamics attack are contained in span{xo) i.e., x^. G span{xo) \/k > 0. 



PROOF. Consider the zero-dynamics attack parameterized by xq and g and denote L as a map for which Lxq = g. 
Then from (17) we have {i^I — {A + BL)) xq = and conclude that xq is an eigenvector of A + BL associated with its 
eigenvalue ly. Now consider the state evolution under attack, x"^^-^ = Ax"^ + Bg with Xq — xq. The proof is completed 
by noting that xf = Axq + Bg = (A + BL)xq = vxq and applying an induction argument. 

Attack performance: Note that the zero-dynamics attack is 0— stealthy only if = x^. However the initial state 
of the system under attack Xq is defined to be zero at the beginning of the attack. Therefore stealthiness of the 

attack may be violated for large differences between = and xq. We refer the reader to [29] for a detailed analysis 
of the effects of zero initial conditions on zero-dynamics attacks. 

If the zero is stable, that is \v\ < 1, the attack will asymptotically decay to zero, thus having little effect on the 
plant. However, in the case of unstable zeros the attack grows geometrically, which could cause a great damage to 
the process. This statement is captured in the following result. 

Theorem 8 A zero- dynamics attack with l^"! > 1 leads the system to an unsafe state if and only if span{xo) is not 
contained in Sx . 



PROOF. Follows directly from Lemma 7 and from the fact that the zero-attack with > 1 generates an unstable 
state trajectory moving away from the origin along span(a;o). 

Disclosure resources: This attack scenario considers an open-loop attack policy and so no disclosure capabilities 
are required, resulting in TZ" = TZ^ = and I^f = = Vfc. 

Disruption resources: The disruption capabilities in this attack scenario correspond to the ability of performing 
deception attacks on the actuator data channels. Therefore the required resources are TZJ = {1, . . . , g}, 7?.j = 0, and 
F = Q 

System knowledge: The ability to compute the open-loop attack policy requires the perfect knowledge zero- 
dynamics, which we denote as tCz- Note that computing the zero-dynamics requires perfect knowledge of the plant 
dynamics, namely A, B, and C. No knowledge of the feedback controller or anomaly detector is assumed in this 
scenario. 

4.5 Local Zero-Dynamics Attack 

In the previous scenario the zero-dynamics attack was characterized in terms of the entire system. Here we further 
restrict the adversary resources by considering that the adversary has disruption resources and knows the model of 
only a subset of the system. In particular, we rewrite the plant dynamics (16) as 



4+1 




^11 A12 




"4" 


+ 


'Bi 






A^X A22_ 




A. 








Ci C2 



(18) 



and assume the adversary has access to only An, A21, Bi, and Ci. Prom the adversary's view, this local system is 
characterized by 

x\+i = Aiix\ + BiOk + Ai2xl 



Vk 



A21 
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where encodes the measurements depending on the local state, C\x]., and the interaction between the local 

subsystem and the remaining subsystems, A2ix\.. 

Attack policy: Similar to the zero-dynamics attack, the attack policy is given by the sequence 

ak = gt^'', 

where g is the input zero direction for the chosen zero u. The input zero direction can be obtained by solving the 
following equation 

J// -All -Bi 

Ci 
_ A21 

Note that the zero-dynamics parameterized by and v correspond to local zero-dynamics of the global system. 

Attack performance: A similar discussion as for the global zero-dynamics attack applies to this scenario. In 
particular, the stealthiness of the local zero-dynamics attack may be violated for large differences between Xq and 
0. Additionally, as stated in Theorem 8, attacks associated with unstable zeros yielding > 1 are more dangerous 
and may lead the system to an unsafe state. 

Disclosure resources: This attack scenario considers an open-loop attack policy and so no disclosure capabilities 
are required, resulting in TZ^ ='Ry = IJ) and = = VA;. 

Disruption resources: The disruption capabilities in this attack scenario correspond to the ability of performing 
deception attacks on the actuator data channels of the local subsystem. Therefore the required resources are TZj = 
{l,...,Q'i},7e?=0, andF = 0. 

System knowledge: The open-loop attack policy requires the perfect knowledge of the local zero-dynamics, 
denoted as /C^ and obtained from An, Bi, Ci, and ^21- 

4-6 Bias Injection Attack 

Here a particular scenario of false-data injection is considered, where the adversary's goal is to inject a constant 
bias in the system without being detected. For this scenario, the class of a— stealthy attacks is characterized at 
steady-state and a method to evaluate the corresponding impact is proposed. Furthermore, we derive the policy 

yielding the largest impact on the system. 

Attack policy: The bias injection attack is illustrated in Figure 5. The attack policy is composed of a steady-state 
component, the desired bias denoted as Uoo, and a transient component. For the transient, we consider that the 

adversary uses a linear low-pass filter so that the data corruptions are slowly converging to the steady-state values. 
As an example, for a set of identical first-order filters the open-loop attack sequence is described by 

afc+i = /3afc + (1 - (3)al,, (19) 

where oq = and < /3 < 1 can be chosen using the results from Theorem 15. The steady-state attack policy yielding 
the maximum impact on the physical system is described below, where the computation of aoo is summarized in 
Theorem 12 and Theorem 14. 

Attack performance: First the steady-state policy is considered. Denote as the bias to be injected and recall 
the anomaly detector dynamics under attack (7). The steady-state detectability of the attack is then dependent on 
the steady-state value of the residual 

= (Ce(/ - Ae)-^Be + De) floo =: Grafloo- 

The largest a— stealthy attacks are then characterized by 

(20) 
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Figure 5. Schematic of the bias injection attack. 

Although attacks satisfying (20) could be detected during the transient, incipient attack signals slowly converging 
to ttoo may go undetected, as it is stated in Theorem 15 and shown in the experiments in Section 5. 

The impact of such attacks can be evaluated using the closed-loop dynamics under attack given by (6). Recalling 



that rk = [xt zl 



, the steady-state impact on the state is given by 

= [/ 0] (7 - A)-^ Bttcx, =: G^afloo. 



Consider the following safe set defined in terms of a;^ . 
Definition 9 The 2— norm safe set S^a is defined as 

Sl. = {xeR-: \\x\\l<l}, 
and the system is said to be in a safe state if x^. G S^a ■ 

For the 2— norm safe set <S|a, the most dangerous bias injection attack corresponds to the a— stealthy attack yielding 
the largest bias in the 2— norm sense, which can be computed by solving 



max||Ga:aaoo||2 

too 

S.t. \\Graaoo\\l < 

Lemma 10 The optimization problem (21) is bounded if and only ifkev{Gra) C ker(Ga;a). 



(21) 



PROOF. Suppose that kcr(G,a) 7^ and consider the subset of solutions where Ooo G ker(Gra)- For this subset 
of solutions, the optimization problem then becomes maX(j^gi5or(Gra) IIGa^aOoo ||2- Since the latter corresponds to 
a maximization of a convex function, its solution is unbounded unless Gxadao = for all a^o G ker(Gro) i-e., 
ker(Gr.o) C ker(Ga;a)- Noting that the feasible set and the objective function are bounded for all solutions Ooo ^ 
ker (Gra) concludes the proof. 

Given Lemma 10, below we consider the non-trivial case for which it holds that ker(Gro) ^ ker(Ga;a)- The above 
optimization problem can be transformed into a generalized eigenvalue problem and the corresponding optimal 
solution characterized in terms of generalized eigenvalues and eigenvectors. Before formalizing this statement, we 
introduce the following result. 



Lemma 11 Let Q £ M"^" and P £ M"^" be positive semi-definite matrices satisfying kei{Q) C ker(P). Denote 
A* as the largest generalized eigenvalue of the matrix pencil (P, Q) and v* as the corresponding eigenvector. Then 
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the matrix P — XQ is negative semi-definite for a generalized eigenvalue X if and only if X = X* . Moreover, we have 
A* > and x^{P — X*Q)x = with Qx if and only if x & span(v*). 

PROOF. The proof can be found in Appendix A. 

The optimal bias injection attack in the sense of (21) is characterized by the following result. 

Theorem 12 Consider the 2— norm safe set S^a and the corresponding optimal a— stealthy bias injection attack 
parameterized by the optimization problem (21), which is assumed to be bounded. Denote X* and v* as the largest 
generalized eigenvalue and corresponding unit-norm eigenvector of the m,atrix pencil {Gj^^Gxa, Gj^^Gra)- The optimal 
bias injection attack is given by 

«:o = ±^7%^^% (22) 

W^raV ||2 

and the corresponding optimal value is HGxaaooHi = ^*^^- Moreover, at steady-state the system is in a safe state if 
and only if X*S^ < 1. 

PROOF. Let P, Q <E R"^" be positive semi-definite matrices such that ker((5) C ker(P). Recall that A is a 
generalized oigonvahic of (_P, Q) if rank(P — XQ) < norniah'ank(P, Q), whore normaJrank(P. Q) is defined as the rank 
of P — vQ for almost all values oiv & C. Furthermore, denote v as the generalized eigenvector associated with A for 
which {P — XQ)v = with v ^ ker(Q). The necessary and sufficient conditions for the optimization problem (21) 
are given by [13] 

= {G^aGxa — ^*Gj^Gra)0'lo, 
n ,i*T^/^T ^ n* — li'^ 

> y^(Gj„G,„ - X*Gj,Gra)y, for y ^ 0. 

Suppose A* is the largest generalized eigenvalue of {Gj^^Gxa, Gj^^Gra) and let v* be the corresponding eigenvector. 

Scaling v* by k so that a^^ = kv* satisfies HGraa^Hl = leads to k = ± ^"yy ^ and the first and second 

conditions are satisfied. As for the third condition, note that Gj^Gxa — ^*Gj^Gra is negative semi-definite by 
Lemma 11, given that A* is the largest generalized eigenvalue, Gl^Gxa and Gj^Gra are positive semi-definite, 

and the assumption that kcr(G'ra) Q ker(G'xa)- To conclude our proof, observe that the optimal value is given by 
al^Gl^Gxaal, = X*al^Gj,Graal, = X*dl = \\x-J\l and thus, by definition, e S^. if and only if A*^^ < i_ 

More generally, the optimal bias injection attacks for ellipsoidal safe sets of the form <Sa;<« = |a;" e M" : x""^ Px"" < iji 
with P positive definite, can be found by replacing the objective function in (21) by IIP^^^GxaOooHi- 

Similarly, consider the safe set as defined below. 

Definition 13 The infinity-norm safe set is defined as 

S^.={x€R^: ||a;|U<l}, 

and the system is said to be in a safe state if x% G . 

Given the infinity-norm safe set , the bias injection attack with the largest impact corresponds to the a— stealthy 
attack yielding the largest bias in the infinity-norm sense. This attack can be obtained by solving the following 
optimization problem 



max IIGxadcx)! 



oo 



s.t. II Gt-qCIqq II2 — * 

A possible method to solve this problem is to observe that 

II GajQ^Qo II QQ — max ||6^ Ga:a^oo||2) 



(23) 
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where the vector is i—th cohimn of the identity matrix. Thus one can transform the optimization problem (23) 
into a set of problems with the same structure as (21), obtaining 



max max llejGxaa^^lL 
' < (24) 

S.t. II 0^0^^00 112 — 

Theorem 14 Consider the infinity-norm safe setS^ and the corresponding optimal a— stealthy bias injection attack 
parameterized by the optimization problem (23), which is assumed to be bounded. Let Ci be the i—th column of the 
identity matrix and denote A* and v* as the largest generalized eigenvalue and corresponding unit-norm eigenvector of 
the matrix pencil Gj^^eieJ G^a — ^Gj^Gra- Letting X* = max^ A*, with v* as the corresponding generalized eigenvector, 
the optimal bias attack is given by 

C = ± 11^'^°. II V*, (25) 

II O-ra't' II 2 

and the corresponding optimal value is ||Ga;o(icx)||oo = VX*5a- Moreover, at steady-state the system is in a safe state 
if and only if X*d'^ < 1. 



PROOF. The proof follows directly from considering the set of optimization problems in (24) and applying Theo- 
rem 12. 



Note that the steady-state value of the data corruption is not sufficient for the attack to be a— stealthy, since the 

transients are disregarded. In practice, however, it has been observed in the fault diagnosis literature that incipient 
faults with slow dynamics are hard to detect [7]. Therefore the low-pass filter dynamics in the attack policy (19) 
could be designed sufficiently slow as to difficult detection. Below we provide a method to verify whether a given 
filter parameter /3 renders the bias attack a— stealthy. 



Theorem 15 Consider the attack policy Uk+i = 0ak + (1 — /3)a^ with /3 G (0, 1). The residual r^ is characterized 
as the output of the autonomous system 

rk+i = m 
ri = crk 

with 



(26) 





Ae Be 










A = 


/?/ (1 - p)i 
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Ce De 









Moreover, the attack policy is a— stealthy for a given if the following optimization problem admits a solution 



mm 7 



s.t. ^<5i 

P C'^ 
C jl 
A^PA -P^O. 



(27) 



h 0, 



PROOF. The autonomous system is directly obtained by considering the augmented state = [^^|^ ij sj]^ , where 
Ik is the state of the low-pass filter bank and Sfe the integral state initialized at sq = «oo • Given this autonomous 
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system, one observes that the attack is a— stealthy if and only if the corresponding output-peak ||r^||| is bounded 
by for all A; > 0, given the initial condition parameterized by a^. The remainder of the proof follows directly 
from the results in [4] regarding output-peak bounds for autonomous systems. 

However, the output-peak bounds are in general conservative and thus the conditions in the previous theorem are 
only sufhcient. 

Disclosure resources: Similarly to the zero attack, no disclosure capabilities are required for this attack, since 
the attack policy is open-loop. Therefore we have 7?." — TZ^ — and =T^ = %\lk. 

Disruption resources: The biases may be added to both the actuator and sensor data, hence the required resources 
are 7?," C {1, . . . , g}, 7?.^ C {1, . . . Since no physical attack is performed, we have = 0. 

System knowledge: As seen in (21), the open-loop attack policy (19) requires the knowledge of the closed-loop 
system and anomaly detector steady-state gains Gra and Gxa^ which we denoted as /Co as shown in Figure 5. 



5 Experiments 



In this section we present our testbed and report experiments on staged cyber attacks following the different scenarios 
described in the previous section. 



5.1 Quadruple- Tank Process 



Our testbed consists of a Quadruple- Tank Process (QTP) [15] controlled through a wireless communication network, 
as shown in Figure 6. 




Centralized controller 



Figure 6. Schematic diagram of the testbed with the Quadruple- Tank Process and a multi-hop communication network. 



The plant model can be found in [15] 



hi = -^y/2ghi + ^^2gh^ + ^^^"i, 

Ai Ai Ai 

h2 = -^y/2gh^+ ^y/2ghi+ ^^^U2, 



^3 ^3 



Oi rz—r- , (1 - ll)k 



1 



-Ml, 



where hi e [0, 30] are the heights of water in each tank, Ai the cross-section area of the tanks, ai the cross-section 
area of the outlet hole, ki the pump constants, 7i the flow ratios and g the gravity acceleration. The nonlinear plant 
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Figure 7. Results for the DoS attack performed against both sensors since t ~ 100 s. 

model is linearized for a given operating point. Moreover, given the range of the water levels, the following safe set 
is considered Sx — {x E R" : \\x — crl||oo < 15, cr = 15}, where 1 e M" is a vector with all entries set to 1. 

The QTP is controlled using a centralized LQG controller with integral action running in a remote computer and 
a wireless network is used for the communications. A Kalman-filter-based anomaly detector is also running in the 
remote computer and alarms are triggered according to (4), for which we computed 5^ — 0.15 and chose Sa = 0.25 
for illustration purposes. The communication network is multi-hop, having one additional wireless device relaying 
the data, as illustrated in Figure 6. 

5.2 Denial- of- Service Attack 

Here we consider the case where the QTP suffers a DoS attack on both sensors, while operating at a constant 
set-point. The state and residual trajectories from this experiment are presented in Figure 7. The DoS attack follows 
a Bernoulli model [1] with p = 0.9 as the probability of packet loss and the last received data is used in the absence 
of data. From Proposition 5, we have that the closed-loop system under such DoS attack is exponentially stable. 

The DoS attack initiates at t « 100 s, leading to an increase in the residual due to successive packet losses. However 
the residual remained below the threshold during the attack and there were no significant changes in the system's 
state. 

5.3 Replay Attack 

In this scenario, the QTP is operating at a constant set-point while a hacker desires to steal water from tank 4, the 
upper tank on the right side. An example of this attack is presented in Figure 8, where the replay attack policy is the 
one described in Section 4.3. The adversary starts by replaying past data from j/2 at t « 90 s and then begins stealing 
water from tank 4 at t ~ 100 s. Tank 4 is successfully emptied and the attacks stops removing water at i « 180 s. 
To ensure stealthiness, the replay attack continues until the system recovered its original setpoint at i « 280 s. As 
we can see, the residue stays below the alarm threshold and therefore the attack is not detected. 
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Figure 8. Results for the replay attack performed against sensor 2 from t ~ 90s to t ~ 280s. Additionally, the adversary 
opens the tap of tank 4 at f ~ 100 s and closes it at f ~ 180 s. 

5.4 Zero- Dynamics Attack 

The QTP has a non-minimum phase configuration in which the plant possesses an unstable zero. In this case, as 
discussed in Section 4.4, an adversary able to corrupt all the actuator channels may launch a false-data injection 
attack where the false-data follows the zero-dynamics. Moreover, since the safe region is described by the set Sx — 
{x G K" : \\x — crl||oo < 15, cr = 15}, from Theorem 8 we expect that the zero-dynamics attack associated with the 
unstable zero can drive the system to an unsafe region. This scenario is illustrated in Figure 9. 

The adversary's goal is to either empty or overflow at least one of the tanks, considered as an unsafe state. The 
attack on both actuators begins at t w 30 s, causing a slight increase in the residual. Tank 3 becomes empty at 
f « 55 s and shortly after actuator 2 saturates, producing a steep increase in the residual which then crosses the 
threshold. However, note that the residual was below the threshold when the unsafe state was reached. 

After saturation of the water level and the actuators, the system dynamics change and therefore the attack signal no 
longer corresponds to the zero-dynamics and is detected, although it has already damaged the system. Thus these 
attacks are particularly dangerous in processes that have unstable zero-dynamics and in which the actuators are 
over-dimensioned, allowing the adversary to perform longer attacks before saturating. 

5.5 Bias Injection Attack 

The results for the case where ui and yi are respectively corrupted with 6^ and 6^ are presented in the Figure 10. In 
this scenario, the adversary aimed at driving the system out of the safe set Sx while remaining stealthy for Sa — 0.25. 
The bias was slowly injected using a first-order low-pass filter with /3 — 0.95 and the following steady-state value, 
computed using Theorem 14, = [6^ 6^]^ = [2.15 - 9.42]^. 

The bias injection began at t w 70s and led to an overflow in tank 4 at i « 225s. At that point, the adversary 
started removing the bias and the system recovered the original setpoint at i w 350 s. The residual remained within 
the allowable bounds throughout the attack, thus the attack was not detected. 
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Figure 9. Results for the zero-dynamics attack starting at f ~ 30 s. Tank 3 is emptied at t ~ 55 s, resulting in a steep increase 
in the residual since the linearized model is no longer valid. 

6 Conclusions 



In this paper we have analyzed the security of networked control systems. A novel attack space based on the 
adversary's system knowledge, disclosure, and disruption resources was proposed and the corresponding adversary 
model described. Attack scenarios corresponding to replay, zero-dynamics, and bias injection attacks were analyzed 
using this framework. In particular the maximum impact of stealthy bias injection attacks was derived and it 
was shown that the corresponding policy does not require perfect model knowledge. These attack scenarios were 
illustrated using an experimental setup based on a quadruple-tank process controlled over a wireless network. 
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A Proof of Lemma 11 



Recall that A is a generalized eigenvalue of (P, Q) if rank(P — XQ) < normalrank(P, Q), where nornialrank(P, Q) is 
defined as the rank of P — i^Q for almost all values of v G C. Furthermore, denote v as the generalized eigenvector 
associated with A for which (P — XQ)v = with v ^ ker{Q). 

Define T = [V^^Vn] € M"^" where the columns of Vn are a basis for ker((5) and VJ^ is chosen such that T is 
nonsingular. Given that ker((5) C ker(P), the coordinate transformation induced by T leads to 



T(P - XQ)T-^ = 



P-XQ 




where Q and P ^ and we conclude that P — XQ ^ if and only if P — XQ_< 0. Additionally, we see that 
all the non-zero generalized eigenvalues of (P, Q) need to reduce the rank of P — XQ and thus need to be positive. 
Hence we have proved that all generalized eigenvalues are non- negative and that A* > 0. 

Now we show that P — XQ is indefinite for all generalized eigenvalues < A <_ A*. Let A > be a generalized 
eigenvalue of (P, Q) with the associated eigenvector v. Then v^{P — XQ)v = (A — X)v^Qv, which can be made 
positive or negative for all generalized eigenvalues A G (0, A*) and thus our assertion is proved. 



As the next step, we show that P — X*Q ^ 0. Since Q is invertible, the generalized eigenvalues of (P, Q) correspond 
to the eigenvalues of the positive semi-definite matrix MPM with M = Q~^/^. Furthermore note that P — X*Q ^ 
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is equivalent to having MPM — \*I which holds since MPM is positive semi-definite with A* as the largest 
eigenvalue. 

All that is left to show now is that (P — X*Q)x = with Qx 7^ if and only if x G span(i;*). Given the condition 
Qx 7^ 0, it is enough to verify that x^ {P — X*Q)x = for x 7^ if and only if x G span(i}*), where v* is the 
generalized eigenvector of {P,Q) associated with A*. The proof is concluded by recalling that P — X*Q ^ 0, hence 
x'^{P — X*Q)x = if and only if x belongs to the subspace spanned by the eigenvectors associated with A*. 
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